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GAO 


United  States  Government  Accountability  Office 
Washington,  DC  20548 


December  21,  2006 

The  Honorable  C.  W.  Bill  Young 
Chairman 

The  Honorable  John  P.  Murtha 
Ranking  Minority  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 

Subject:  Defense  Contracting — Questions  for  the  Record 

On  September  7,  2006, 1  testified  before  the  Subcommittee  on  recent  trends  in  Department  of 
Defense  (DOD)  contracting.  Specifically,  I  testified  about  practices  that  undermine  DOD’s 
ability  to  establish  sound  business  arrangements,  particularly  those  involving  the  selection  and 
oversight  of  DOD’s  contractors  and  their  performance.  The  need  for  prudence  with  taxpayer 
funds  and  long-range  fiscal  challenges  demand  that  DOD  ensure  that  its  funds  are  spent  wisely, 
and  that  it  is  buying  the  right  things,  the  right  way. 

The  Subcommittee  requested  that  I  respond  to  a  number  of  post-hearing  questions  relating  to 
various  issues,  including  measures  that  DOD  can  employ  to  ensure  better  contracting 
outcomes.  The  specific  questions  and  my  responses  are  attached  in  appendix  I.  The  responses 
are  based  on  previously  issued  GAO  products  on  DOD  acquisitions,  all  of  which  were 
conducted  in  accordance  with  generally  accepted  government  auditing  standards.  Because  the 
responses  are  based  on  prior  work,  we  did  not  obtain  comments  from  DOD. 

We  will  make  copies  of  this  letter  available  to  others  upon  request,  and  it  will  be  available  at  no 
charge  on  the  GAO  Web  site  at  http://www.gao.gov.  If  you  have  any  questions  about  this  letter 
or  need  additional  information  please  contact  Katherine  V.  Schinasi  at  (202)  512-4841  or 
schinasik@ gao .  gov.  Contact  points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs 
may  be  found  on  the  last  page  of  this  letter.  Key  contributors  to  this  letter  were  Carol  Dawn 
Petersen,  Chris  Kunitz,  and  Bill  Woods. 


Sincerely, 


David  M.  Walker 
Comptroller  General 
of  the  United  States 
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Appendix  I:  GAO  Responses  to  Questions  for  the  Record 


Award  and  Incentive  Fees 


1.  Could  you  address  how  these  award  and  incentive  fees  work,  in  practical  terms? 

When  we  talk  about  fees,  we  are  really  talking  about  a  government  expense  and  a  form  of 
profit  for  contractors.  Depending  on  how  a  contract  is  structured,  award  and  incentive  fees 
can  account  for  all  or  a  portion  of  a  contractor’s  potential  fee  or  profit.  Unlike  other 
contracts,  award-  and  incentive-fee  contracts  allow  an  agency  to  adjust  the  amount  of  fee 
paid  to  contractors  based  on  the  contractors’  performance.  In  practical  terms,  an  award-fee 
contract  sets  up  a  system  where  the  government  periodically  assesses  the  contractor’s 
performance  in  areas  such  as  quality,  timeliness,  technical  ingenuity,  and  cost-effective 
management.  These  criteria  are  generally  subjective  and  can  include  inputs  as  well  as 
outcomes.  Incentive  fees,  however,  are  more  objective  and  outcome-based.  The  contract 
identifies  specific  cost  or  delivery  targets,  and/or  performance  goals,  and  the  amount  of  fee 
earned  by  the  contractor  is  to  be  directly  related  to  achievement  of  these  goals. 

2.  Do  you  believe  they  are  an  effective  way  to  create  incentives  with  contractors?  In 
other  words,  do  they  improve  contractor  performance? 

Award  and  incentive  fees  can  be  an  effective  tool  if  properly  designed  and  effectively 
administered.  However,  for  these  fees  to  be  effective,  DOD  must  address  the  underlying 
problems  in  its  acquisition  system  and  more  directly  link  the  fees  to  the  outcomes  it  wants. 
Because  the  weapon  system  programs  that  result  from  this  system  are  in  many  cases 
unexecutable  and/or  subject  to  changing  “requirements”  or  funding  levels,  DOD  has  been 
unwilling  to  hold  its  programs  or  its  contractors  accountable  for  achieving  the  very 
acquisition  outcomes  it  has  identified.  As  a  result,  fees  are  paid  even  when  outcomes  do  not 
meet  expectations.  Addressing  these  broader  acquisition  issues  and  strengthening  the  link 
between  fees  and  acquisition  outcomes  can  increase  the  accountability  of  DOD  programs  for 
fees  paid,  of  contractors  for  results  achieved,  and  the  likelihood  that  these  fees  will  motivate 
the  contractors  and  be  an  effective  tool  for  the  government. 

3.  Do  you  believe  we  should  discontinue  the  use  of  award  and  incentive  fees? 

No,  we  do  not  believe  they  should  be  discontinued.  Award  and  incentive  fees  can  be  useful  if 
they  are  used  in  the  appropriate  setting.  Each  contract  type  has  a  use  based  on  the  level  of 
risk  involved.  The  problem  occurs  when  you  proceed  into  programs  without  realistic 
requirements  and  sufficiently  mature  technologies  on  which  to  base  realistic  cost  and 
schedule  estimates  and  attempt  to  offset  that  increased  risk  by  offering  award  fees  to 
motivate  the  contractor  to  overcome  that  risk.  The  important  question  is:  Have  you 
adequately  defined  and  established  appropriate  criteria  that  enable  you  to  measure 
outcomes?  And  finally,  how  do  you  apply  those  criteria  in  determining  the  level  of  fee  that 
can  be  justified?  We  have  made  several  recommendations  to  this  effect,  and  DOD  has 
responded  favorably  with  new  guidance  to  link  award  fees  to  acquisition  outcomes. 

However,  as  with  other  recommendations  we  have  made  related  to  DOD  weapon  system 
acquisitions,  the  key  will  be  how  this  new  policy  is  implemented. 
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Acquisition  Policy 


1.  Given  that  DOD’s  acquisition  policy  seems  to  embrace  practices  endorsed  by  the 
Defense  Acquisition  Performance  Assessment,  the  Defense  Science  Board  Summer 
Study  2005,  and  GAO,  what  is  preventing  the  services  from  consistently  executing 
these  policies? 

The  services  are  not  consistently  executing  these  policies  because  DOD  is  not  enforcing  a 
knowledge-based  approach,  discipline  is  lacking,  and  business  cases  do  not  measure  up.  This 
is  occurring,  in  part,  because  there  are  no  consequences  for  actions  that  run  counter  to  the 
intent  of  DOD  acquisition  polices — officials  responsible  for  approving  program  starts  are  no 
longer  in  their  positions  by  the  time  the  consequences  of  their  actions  become  evident.  The 
department  routinely  accepts  high  levels  of  technology  risk  at  the  start  of  major  acquisition 
programs.  Mature  technologies  are  pivotal  to  developing  new  products.  Without  mature 
technologies  at  the  outset,  a  program  will  almost  certainly  incur  cost  and  schedule  problems. 
However,  DOD’s  acquisition  community  moves  forward  on  programs  with  technologies 
before  they  are  mature  and  takes  on  responsibility  for  technology  development  and  product 
development  concurrently.  Our  work  has  also  shown  that  DOD  allows  programs  to  begin 
without  establishing  a  sound  business  case  that  matches  requirements  with  technology, 
acquisition  strategy,  time,  and  funding.  And  once  these  programs  begin,  their  requirements 
and  funding  change  over  time.  In  fact,  program  managers  consider  shifting  requirements — 
which  can  result  in  added  program  complexity  and  costs — and  funding  instabilities — which 
occur  throughout  the  program — to  be  their  biggest  obstacles  to  success.  Fundamentally, 

DOD  will  need  to  reexamine  the  entirety  of  its  acquisition  process  and  how  it  is  affected  by 
requirements  and  funding  processes.  This  includes  making  significant  changes  to  program 
requirements  setting,  funding,  and  execution. 

2.  What  else  is  needed  to  ensure  that  cost,  schedule,  and  performance  outcomes  for 
new  weapons  are  predictable  and  achievable  when  these  programs  seek  approval 
from  Congress? 

DOD  needs  to  take  additional  steps  to  achieve  outcomes  on  par  with  best  practices.  These 
include 

•  developing  and  implementing  an  acquisition  investment  strategy; 

•  ensuring  that  individual  programs  are  executable;  and 

•  clearly  delineating  responsibilities  and  holding  government  employees  and  contractors 
accountable  for  achieving  desired  results. 

While  DOD  has  incorporated  into  policy  a  framework  that  supports  a  knowledge-based 
acquisition  process  similar  to  that  used  by  leading  organizations,  it  must  establish  stronger 
controls  to  ensure  that  decisions  on  individual  programs  are  informed  by  demonstrated 
knowledge.  Moreover,  Congressional  approval  of  programs  that  have  not  taken  these  steps 
encourages  DOD’s  subsequent  requests  for  additional  funding. 
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3.  What  are  the  services  doing  now  to  ensure  that  the  technologies  in  these  programs 
are  achieving  the  desired  level  of  maturity  before  they  proceed  into  system 
development? 

DOD  is  now  required  by  law  to  certify  to  the  Congress  that  technology  is  demonstrated  to  a 
specific  maturity  level  before  being  approved  for  system  development.  However,  this  is  a 
recent  statutory  requirement  and  it  is  too  early  to  tell  whether  this  requirement  will  ensure 
achievement  of  the  desired  technology  maturity  level  before  programs  are  authorized  to 
proceed  into  system  development.  Prior  to  this  new  law,  some  of  the  criteria  upon  which  this 
certification  is  based  were  reflected  in  DOD  policy.  However,  our  prior  work  showed  that 
DOD  written  polices  are  not  always  observed  in  practice. 

4.  GAO  believes  a  sound  business  case  is  needed  before  a  weapon  system  begins 
system  development  and  demonstration  yet  finds  many  of  the  existing  weapons’ 
business  cases  to  be  broken.  What  do  you  consider  key  elements  of  a  business  case 
for  starting  a  development  program  and  how  do  you  ensure  these  are  followed  in 
each  planned  development  program? 

The  following  are  the  essential  elements  of  a  sound  business  case: 

•  A  requirement  exists  that  warrants  a  solution  consistent  with  national  security  priorities. 

•  The  developer  has  the  mature  technologies  and  technical  knowledge  necessary  to  meet 
the  requirement. 

•  The  developer  has  a  knowledge-based  product  development  plan  that  will  attain  high 
levels  of  design  and  production  maturity  at  the  right  times. 

•  Reasonable  estimates  have  been  developed  to  execute  the  product  development  and 
production  plan. 

•  Current  and  future  funding  will  be  available  to  fully  resource  the  product  development 
and  production  plan. 

•  The  agency  has  the  capacity,  either  in-house  or  by  contractual  arrangement,  to  manage  its 
programs,  including  ensuring  that  contractors  are  subject  to  appropriate  independent 
oversight  as  to  cost,  schedule,  and  quality. 

To  ensure  that  these  elements  are  in  place  at  the  individual  program  level,  DOD  needs  to 
(1)  begin  programs  in  the  context  of  an  overall  funding-constrained,  weapon  system 
portfolio,  and  (2)  say  “no”  when  individual  programs  do  not  measure  up.  DOD  will  need  the 
support  of  the  Congress  to  succeed. 
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Management  of  Technological  Risk 


1.  It  has  often  been  demonstrated  that  concurrent  technology  development  and 
production  in  programs  lead  to  wasted  funds  and  undelivered  capability.  Do  you 
share  in  this  concern?  Please  elaborate. 

I  do  share  in  this  concern.  It  is  essential  that  technology  be  developed  separately — and 
before — t  he  development  phase  of  a  product  or  weapon  system.  This  is  a  best  practice  that 
results  in  predictable  program  outcomes.  Our  work  shows  programs  that  proceed  with 
mature  technology  average  about  5  percent  cost  growth;  programs  with  immature  technology 
experience  about  35  percent  cost  growth.  Moreover,  immature  technology  delays  design  and 
production  maturity. 

2.  Your  most  recent  annual  report  on  DOD  weapon  systems  has  many  cases  in  which 
programs  are  entering  production  before  TRL  Level  7  has  been  reached.  Can  you 
offer  some  examples  of  programs  with  unnecessary  concurrency? 

As  we  noted  in  that  report  ( GAQ-06-39 11.  allowing  technology  development  to  spill  over  into 
product  development  puts  an  extra  burden  on  decision  makers  and  provides  a  weak 
foundation  for  making  product  development  estimates.  We  found  that  programs  that  began 
with  immature  technologies  have  experienced  average  research  and  development  cost 
growth  of  34.9  percent;  programs  that  began  with  mature  technologies  have  only  experienced 
cost  growth  of  4.8  percent.  Examples  of  programs  with  concurrency  include  the  F-22A,  the 
Joint  Strike  Fighter,  the  Future  Combat  Systems,  the  VH-71  Presidential  Helicopter,  the 
Global  Hawk,  the  CVN-21,  the  DDG-1000,  the  Multi-mission  Maritime  Aircraft,  and  the 
National  Polar-orbiting  Operational  Environmental  Satellite  System,  among  others. 

3.  What  prevents  service  acquisition  executives  from  demanding  that  acquisition 
programs  not  go  forward  until  technology  is  mature? 

First,  the  budgeting  process  requires  that  funding  for  a  program  is  put  on  the  table  2  to 
3  years  in  advance — this  creates  pressure  to  proceed  with  the  program  despite  how  mature 
the  technology  is.  Second,  the  requirements  process  tends  to  settle  on  ultimate  performance, 
which  puts  pressure  on  programs  to  reach  for  exotic  technology.  Third,  it  is  easier  to  say 
“yes”  than  “no”.  There  can  always  be  a  compelling  case  made  that  a  particular  program  is  an 
exception;  the  problem  is  that  there  are  way  too  many  exceptions.  And  finally,  this  is 
occurring,  in  part,  because  there  are  no  consequences  for  actions  that  run  counter  to  the 
intent  of  DOD  acquisition  polices — officials  responsible  for  approving  programs  are  no 
longer  in  their  positions  by  the  time  the  consequences  of  their  actions  become  evident. 

4.  Should  concurrent  technology  development  and  system  production  be  an  exception 
rather  than  the  rule? 

Definitely.  Sequential  phasing  of  technology  development  and  system  development  is  DOD 
policy,  best  practice,  and  essential  to  delivering  capability  faster  to  the  warfighter  and  more 
economically  to  the  taxpayer. 
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5.  What  concrete  proposals  should  be  put  in  place  to  ensure  programs  are  not  allowed 
to  proceed  to  production  without  demonstrated  maturity  in  technology? 


DOD  needs  to  make  sure  each  new  program  is  executable  before  the  development  begins, 
and  well  before  production  is  considered.  For  example, 

•  Science  and  technology  organizations  need  the  resources,  authority,  and  tools  to  mature 
technology  and  transition  it  to  acquisition  programs. 

•  Requirements  must  be  clearly  defined  and  achievable  given  available  resources,  including 
beginning  product  development  with  mature  technologies.  Once  begun,  requirements 
should  not  change  without  assessing  their  potential  to  disrupt  the  program  and  without 
being  fully  resourced. 

•  DOD  should  say  no  to  programs  that  do  not  measure  up,  including  canceling  programs 
when  appropriate. 

•  If  a  decision  is  made  to  proceed  with  immature  technologies,  the  cost  and  schedule 
estimates  should  be  increased  commensurate  with  the  additional  risk. 

In  addition,  the  Congress  can  help  by  not  funding  programs  that  do  not  measure  up. 


Role  of  Contractors 


1.  Would  you  agree  that  DOD  is  relying  more  heavily  on  contractors  to  manage  and 
deliver  weapon  systems? 

It  is  clear  that  DOD  continues  to  increase  its  reliance  on  contractors  to  provide  both  products 
and  services.  DOD’s  obligations  on  service  contracts  rose  from  $82.3  billion  in  fiscal  year 
1996  to  $141.2  billion  in  fiscal  year  2005.  Based  on  our  work  looking  at  various  major 
systems,  we  have  observed  how  DOD  is  relying  on  contractors  in  new  ways  to  manage  and 
deliver  weapon  systems.  For  example,  the  Army’s  Future  Combat  Systems  is  using  a  lead 
system  integrator  approach  in  which  the  prime  contractor  has  greater  than  usual 
responsibilities  in  areas  such  as  requirements  definition,  system  design,  and  selection  of 
major  subsystem  contractors.  This  could  increase  taxpayer  risk,  especially  if  the  government 
does  not  provide  adequate  oversight. 

2.  What  factors  are  contributing  to  this  increased  dependence  on  contractors? 

A  variety  of  factors  have  contributed  to  the  increased  dependence  on  contractors.  In  the  case 
of  the  Future  Combat  Systems  program,  for  example,  the  Army  chose  to  use  a  lead  systems 
integrator  because  it  did  not  believe  it  had  the  in-house  resources  or  flexibility  to  field  such  a 
complex  system  in  the  time  required.  More  generally,  spending  on  service  contracting  has 
increased  in  part  to  compensate  for  a  declining  civilian  workforce.  Increased  security 
requirements  since  September  1 1  have  also  led  to  more  contract  spending,  particularly  in 
areas  such  as  guard  services. 
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3.  What  risks  does  it  pose  to  protecting  the  government’s  interests? 

As  reliance  on  contractors  has  increased,  so  has  the  need  to  ensure  that  the  government’s 
interests  are  protected.  Organizational  conflicts  of  interest  present  one  type  of  risk  to  the 
government  and  these  must  be  identified  and  managed,  which  requires  careful  government 
oversight.  When  the  government  is  acquiring  services,  challenges  arise  when  attempting  to 
define  requirements,  establish  measurable  and  performance-based  outcomes,  and  assess 
contractor  performance. 

4.  Do  you  think  this  increased  dependence  has  been  a  conscious,  managed  trend,  or 
has  it  more  or  less  happened  to  us? 

In  the  area  of  service  acquisitions  in  particular,  which  currently  exceeds  the  value  of  major 
weapon  systems,  the  growth  in  spending  has  not  been  a  managed  outcome.  For  example, 
when  DOD  reduced  the  size  of  its  civilian  workforce  in  the  1990s,  it  did  so  without 
proactively  shaping  the  workforce  to  ensure  that  it  had  the  specific  skills  and  competencies 
needed  to  accomplish  its  mission.  DOD  has  acknowledged  that  it  faces  significant  workforce 
challenges  that,  if  not  effectively  addressed,  could  impair  the  responsiveness  and  quality  of 
acquisition  outcomes.  In  a  recent  report,  DEFENSE  ACQUISITIONS:  Tailored  Approach 
Needed  to  Improve  Service  Acquisition  Outcomes,  GAO-O 7-20,  Nov.  2006,  we  recommended 
a  number  of  steps  DOD  should  take  to  make  service  acquisitions  a  managed  outcome. 

5.  What  steps  do  we  need  to  take  to  protect  the  government’s  interests  (that  is,  on 
behalf  of  both  the  warfighter  and  the  taxpayer)  in  an  environment  of  increasing 
dependence  on  contractors? 

We  are  currently  reviewing  a  number  of  aspects  of  this  issue,  including  conflict  of  interest 
laws  and  ethical  conduct  standards  for  contractor  employees.  In  addition,  the  Congress  can 
play  an  important  role  through  its  oversight  function  in  making  sure  that  taxpayer  interests 
are  protected.  On  November  17,  2006,  we  offered  a  set  of  issues,  based  on  GAO’s  work,  for 
consideration  for  the  agenda  of  the  110th  Congress  ( Suggested  Areas  for  Oversight  for  the 
110th  Congress,  GAO-Q7-235R).  The  first  set  suggested  targets  for  near-term  oversight, 
including  the  need  to  address  governmentwide  acquisition  and  contracting  issues.  In  this 
regard,  we  noted  that  the  work  of  the  government  is  increasingly  being  performed  by 
contractors  and  many  agencies  rely  on  contractors  to  carry  out  their  basic  missions.  At  the 
same  time,  GAO’s  list  of  government  high-risk  areas  includes  acquisition  and  contract 
management  issues  that  collectively  expose  hundreds  of  billions  of  taxpayer  dollars  to 
potential  waste  and  misuse.  Consequently,  we  suggested  that  Congress  continue  to  monitor 
agencies’  efforts  to  address  existing  problems,  while  facilitating  a  re-examination  of  the  rules 
and  regulations  that  govern  the  government-contractor  relationship  in  an  increasingly 
blended  workforce.  Actions  and  topics  that  need  continued  congressional  oversight  include: 

•  requiring  agencies  to  report  on  mechanisms  in  place  to  ensure  that  contractors  are 

playing  appropriate  roles  and  that  agencies  have  retained  sufficient  workforce  capacity  to 
monitor  contractor  cost,  quality,  and  performance,  particularly  in  such  critical  operations 
as  responding  to  Hurricane  Katrina  and  rebuilding  Iraq; 
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•  assessing  agencies’  efforts  to  ensure  that  acquisitions  are  performance-  and  outcome- 
based,  with  appropriate  risk-sharing  contracts  in  place. 

•  requiring  agencies  with  significant  acquisition  budgets,  such  as  DOD,  to  better  align 
requirements,  budget,  and  acquisition  processes  to  reconcile  the  differences  between 
wants,  needs,  affordability,  and  sustainability,  given  current  and  future  demands  and 
resources;  and 

•  monitoring  the  implementation  of  agency  action  plans  to  address  the  GAO  high-risk  areas 
related  to  acquisition  and  contract  management. 


Incremental  Funding 

1.  Is  it  a  good  idea  to  expand  the  use  of  incremental  funding  for  major  weapon  system 
programs?  If  not,  why  not? 

GAO  has  advocated  full  funding  for  capital  asset  acquisitions  as  a  way  to  increase  recognition 
of  implied  commitments  embodied  in  budgetary  decisions  as  compared  to  the  incremental 
funding  approach.  That  said,  I  am  not  aware  of  any  plans  by  DOD  to  expand  the  use  of 
incremental  funding  for  major  weapon  system  programs.  Nevertheless,  in  testimony  earlier 
this  year  before  the  Airland  Subcommittee  of  the  Senate  Armed  Services  Committee,  I 
expressed  concern  about  an  Air  Force  plan  to  use  incremental  funding  for  a  multiyear 
contract  for  the  F-22A  program  (TACTICAL  AIRCRAFT:  Questions  Concerning  the  F-22A,s 
Business  Case,  GAQ-06-99 IT.  July  25,  2006).  Subsequent  to  that  hearing,  the  Congress 
authorized  multiyear  procurement  for  the  F-22A,  but  expressly  prohibited  the  use  of 
incremental  funding  for  the  contract. 

2.  Is  it  more  dangerous  to  expand  the  use  of  incremental  funding  when  the  overall 
budget  is  projected  to  fall  or  become  more  competitive?  If  so,  why? 

Inherent  in  the  concept  of  incremental  funding  of  a  program  or  project  is  the  expectation  that 
additional  funds  will  be  available  in  subsequent  years  to  bring  the  program  or  project  to 
completion.  Declining  budgets  or  increased  competition  for  available  resources  in 
subsequent  years  would  certainly  make  it  more  difficult  for  that  expectation  to  be  realized. 


NAVY:  I)I)(X)  Destroyer 

1.  What  factors  have  contributed  to  driving  DD(X)  costs  up  and  quantities  down? 

DD(X)  (now  DDG  1000)  lead  ship  procurement  costs  have  increased  to  $3.3  billion  because 
the  Navy  underestimated  the  resources  that  would  be  required  to  construct  (as  well  as 
design)  a  multi-mission  ship  offering  significant  new  capabilities  related  to  air  defense,  land 
attack,  and  undersea  warfare.  In  addition,  the  Navy  did  not  complete  a  comprehensive, 
program  life  cycle  cost  estimate  until  March  2005.  Prior  to  this  estimate,  the  Navy  relied  on 
program  and  contractor  estimates  and  the  life  cycle  cost  estimate  completed  in  1997  for  the 
former  DD  21  program. 
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2.  Is  this  symptomatic  of  DOD’s  requirements,  budgeting,  and  acquisition  processes  in 
general? 

Yes.  Our  work  has  shown  that  DOD  programs  often  cost  more  and  take  longer  to  develop 
than  estimated  when  program  managers  are  forced  to  make  cost  and  schedule  predictions 
based  on  unproven  technologies.  Alternatively,  initiating  product  development  with  a  high 
level  of  knowledge  about  program  technologies  can  help  mitigate  such  risks. 

As  knowledge  has  grown  in  the  DD(X)  program,  the  Navy  has  frequently  discovered  that  its 
desired  capabilities  did  not  align  with  the  funding  it  had  available.  To  compensate,  the  Navy 
has  steadily  reduced  planned  ship  quantities  and  land  attack  capabilities  to  enable  continued 
funding  of  other  technology  demonstration  efforts  on  the  ship. 

We  have  also  pointed  out  that  the  Navy  generally  does  not  hold  a  milestone  B,  which 
approves  entry  into  the  System  Development  and  Demonstration  phase,  for  ship  programs 
until  it  is  authorizing  ship  construction.  Other  DOD  programs  are  required  to  complete 
milestone  B  before  entering  System  Development  and  Demonstration.  Completion  of  an 
independent  cost  estimate  is  tied  to  the  milestone.  As  a  result  the  Navy  invested  billions  in 
DD(X)  research  and  development  without  an  in-depth  analysis  of  cost.  This  practice  of 
delaying  the  milestone  continues  today  on  such  programs  as  Littoral  Combat  Ship  and 
amphibious  assault  ship 
(LHA  6). 

3.  What  are  the  key  risks  that  could  jeopardize  delivering  the  DD(X)  on  time  and 
within  cost? 

The  Navy’s  strategy  to  continue  maturing  DD(X)  technologies  while  refining  the  ship’s  design 
and  beginning  construction  of  lead  ships  places  the  program  at  significant  risk  for  additional 
cost  growth  and  schedule  delays. 

Key  development  risks  include  remaining  work  associated  with  maturing  technologies, 
especially  the  volume  search  radar;  completing  design  of  the  integrated  deckhouse;  and 
maintaining  software  release  schedules. 

Because  the  Navy  plans  to  begin  construction  of  the  lead  DD(X)  ships  with  an  incomplete 
understanding  of  the  ship’s  critical  technologies,  design  changes  may  occur  that  could 
require  costly  rework  throughout  the  construction  phase  for  the  lead  ships. 

4.  What  steps  need  to  be  taken  on  DD(X)  to  ensure  that  it  is  delivered  on  time  and 
within  cost? 

The  Navy’s  cost  estimate  for  DD(X)  assessed  a  45  percent  probability  of  the  lead  ship 
delivering  at  or  below  a  cost  of  $3.3  billion.  Consequently,  the  Navy  may  be  forced  to  revisit 
planned  capabilities  for  the  ship  in  the  event  that  its  $3.3  billion  cost  target  becomes 
unattainable.  Moreover,  before  construction  begins  in  fiscal  year  2008,  the  Navy  should 
ensure  that  all  technologies,  such  as  the  volume  search  radar,  have  been  fully  demonstrated. 
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NAVY/MARINE  CORPS:  Expeditionary  Fighting  Vehicle  fEFVj 


1.  What  went  wrong  on  the  EFV  program  after  it  had  such  a  good  start? 

Program  difficulties  occurred  in  part  because  not  enough  time  was  allowed  to  demonstrate 
maturity  of  the  EFV  design  during  the  System  Development  and  Demonstration  (SDD)  phase. 
Best  practices  and  current  DOD  acquisition  policy  call  for  system  integration  work  to  be 
conducted  before  critical  design  review  is  held.  This  review  represents  the  commitment  to 
building  full-scale  SDD  prototypes  that  are  representative  of  the  production  vehicle.  In  the 
case  of  the  EFV,  however,  the  SDD  critical  design  review  was  held  before  the  system 
integration  work  had  been  completed.  While  testing  of  early  prototypes  began  one  year 
before  the  SDD  critical  design  review,  it  continued  for  three  more  years  after  the  decision  to 
begin  building  the  SDD  prototypes.  Test  schedules  for  demonstrating  design  maturity  proved 
optimistic  and  success-oriented,  and  were  extended,  and  major  problems  were  discovered  in 
testing  of  the  prototypes.  Specifically,  the  original  schedule  did  not  allow  adequate  time  for 
testing,  evaluating  the  results,  fixing  the  problems,  and  retesting  to  make  certain  that 
problems  are  fixed  before  moving  forward.  The  SDD  schedule  of  about  three  years  proved 
too  short  to  conduct  all  necessary  planning  and  to  incorporate  the  results  into  design 
changes,  resulting  in  schedule  delays  and  cost  increases. 

2.  Could  these  problems  have  been  avoided? 

Yes.  The  program  was  positioned  for  a  good  start  in  that  it  had  mature  technologies  fairly 
early.  Subsequent  problems  could  have  been  avoided  if  the  program’s  System  Development 
and  Demonstration  phase  followed  DOD  policy  preferences  and  best  practices  as  they 
related  to  design  and  production  maturity. 

3.  Was  DOD  policy  in  error,  or  was  the  program  not  following  policy? 

DOD  policy  was  not  in  error.  In  fact,  the  EFV  was  following  a  knowledge-based  approach  and 
other  best  practices,  which  had  been  incorporated  into  DOD’s  guidelines  in  2000  for  major 
acquisition  and  expanded  into  its  May  2003  policy.  In  addition  to  having  reached  a  relatively 
high  level  of  technology  maturity  before  starting  SDD,  the  EFV  program  had  earlier  adopted 
best  practices  in  its  implementation  of  Integrated  Product  Teams  and  had  trained  its  program 
office  staff  on  this  acquisition  improvement  initiative.  As  noted  above,  the  EFV  departed  from 
the  tenets  of  DOD  acquisition  policy  and  best  practices  shortly  after  SDD  began,  when  it 
attempted  to  do  system  integration  and  system  demonstration  concurrently. 

4.  How  do  strategies  that  go  against  sound  policy  get  approved? 

Our  work  on  multiple  weapon  systems  shows  that  most  programs  proceed  with  strategies 
that  do  not  comport  with  the  knowledge-based  approach  embodied  in  DOD  written  policies. 
For  example,  in  our  March  2006  assessment  of  weapon  systems,  only  10  percent  of  the 
weapon  systems  reviewed  had  begun  SDD  with  mature  technologies.  There  are  several 
reasons  for  this.  First,  although  DOD  acquisition  policy  allows  individual  programs  to  tailor 
their  own  acquisition  approaches,  it  does  not  include  sufficient  controls  to  ensure  that  key 
aspects  of  the  policy  are  implemented.  Second,  programs  proceed  with  optimistic 
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assumptions  about  what  they  can  accomplish  for  their  estimated  cost  and  schedules.  Third, 
as  weapon  system  programs  proceed  through  development,  the  desire  to  protect  the  program 
against  disruption  intensifies. 

5.  What  lesson  from  the  EFY  experience  can  be  drawn  and  applied  to  DOD  acquisitions 
in  general? 

Using  the  lens  of  a  knowledge-based  business  case,  the  EFY  at  the  start  of  SDD  was  sound  on 
requirements  and  technology  maturity  (knowledge  point  1).  While  design  stability  was  judged 
to  be  attained  at  the  critical  design  review  (knowledge  point  2)  immediately  after  entering 
SDD,  it  appears  that  holding  critical  design  review  so  soon  was  premature.  This  particular 
acquisition  strategy  did  not  provide  the  resources  (time  and  money)  necessary  to 
demonstrate  design  maturity  and  production  maturity  (knowledge  point  3).  A  key  lesson 
learned  is  that  while  it  is  necessary  to  demonstrate  one  knowledge  point  before  a  subsequent 
one  can  be  demonstrated,  this  alone  is  not  sufficient.  Attaining  one  knowledge  point  does  not 
guarantee  the  attainment  of  the  next  one.  Rather,  the  acquisition  strategy  for  any  program 
must  adequately  provide  for  the  attainment  of  each  knowledge  point  even  in  programs,  such 
as  the  EFV,  which  were  in  a  favorable  position  at  the  start  of  SDD.  If  the  acquisition  strategy 
does  not  adequately  provide  for  the  attainment  of  all  knowledge  points,  the  estimates  for  cost 
and  schedule  will  not  have  a  sound  basis. 


NAVY:  Cost  Growth  in  Shipbuilding  Programs 

1.  What  did  the  GAO’s  work  reveal  as  the  major  causes  of  continued  overruns  in  the 
Navy’s  shipbuilding  program? 

Our  February  2005  report  fGAO-Q5-183j  on  cost  growth  in  shipbuilding  programs  cited 
unrealistic  estimates  of  costs  as  a  key  source  of  cost  growth.  Unrealistic  estimates  in  turn  led 
to  unrealistic  budgets  and  contract  prices.  Increases  in  labor  hours  and  material  costs  were 
common,  with  low  technology  maturity  at  program  start  and  immature  design  before 
beginning  production  contributing  to  these.  We  also  identified  deficiencies  in  cost  reporting 
as  a  problem. 

2.  The  Navy  has  pledged  over  the  past  year  or  so  to  address  this  problem.  Have  you 
seen  any  concrete  proposals,  and  if  so,  what  do  you  think  of  them? 

The  Navy  has  cited  several  actions  it  has  taken  to  improve  cost  performance.  These  include 
using  risk  analysis  in  developing  cost  estimates,  budgeting  at  the  Office  of  the  Secretary  of 
Defense  Cost  Analysis  Improvement  Group  estimate,  using  realistic  inflation  rates  in 
developing  cost  estimates,  withholding  fees  if  the  contractor  does  not  provide  useable  and 
timely  cost  reports,  and  introducing  more  aggressive  contract  incentives. 

We  believe  these  are  positive  steps  but  it  remains  to  be  seen  whether  the  Navy  will  follow 
through  with  these  efforts.  For  example,  while  the  Navy  has  said  that  it  will  use  contract 
incentives  in  the  form  of  more  demanding  share  lines,  lower  ceiling  prices,  and  lower 
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minimum  fees  (for  cost  plus  incentive  fee  contracts),  a  key  issue  is  whether  these  policies 
will  be  sustained  over  time. 

We  also  believe  that  further  actions  are  needed.  To  reduce  the  risk  of  cost  growth,  the  Navy 
needs  to  insure  that  technology  risk  is  not  carried  forward  into  construction.  Technology 
risk,  and  with  it  the  potential  for  design  changes,  needs  to  be  addressed  before  construction 
begins.  This  has  not  been  the  case  with  the  DD(X)  program,  where  key  technologies — such 
as  the  dual  band  radar — have  not  been  fully  demonstrated. 

Moreover,  we  believe  the  Navy  could  better  ensure  realism  of  budgets  and  contracts.  We 
believe  funding  and  contracting  for  a  detailed  design  of  the  ship  should  be  separated  from  the 
funding  and  contracting  for  construction.  A  budget  request  for  ship  construction  would  be 
informed  by  the  knowledge  gained  from  the  first  year  or  two  of  detail  design. 

3.  Should  ship  programs  be  put  on  firm  fixed  price  contracts? 

Use  of  fixed  price  contracts  is  dependent  on  how  much  uncertainty  exists.  It  is  important  to 
reduce  uncertainty  early  in  a  program  so  that  technology  and  design  risks  are  not  carried 
forward  into  construction.  Following  a  knowledge-based  approach  will  likely  enable  follow- 
on  ships  to  use  fixed  price  contracts.  Use  of  such  contracts  would  provide  the  strongest 
incentive  for  improving  cost  performance — under  a  fixed  price  contract  the  cost  risk  of 
performance  generally  shifts  to  the  contractor. 

4.  What  specific  ideas  would  you  offer  to  deal  with  this  problem? 

In  addition  to  the  suggestions  discussed  above,  we  believe  the  Navy  should  increase  its  use  of 
cost-performance  trade  studies — including  design-to-cost  and  cost  as  an  independent 
variable  (CATV)  techniques — to  identify  cost  reduction  opportunities  in  its  new  ship  designs 
and  position  the  Navy  to  achieve  “best  value”  outcomes  in  its  shipbuilding  programs. 
Establishing  design-to-cost  goals  early  in  a  program  encourages  consideration  of  cost  as  a 
key  design  parameter  when  evaluating  other  parameters  such  as  schedule,  performance,  and 
operational  capability.  Goals  can  be  set  in  each  phase  of  the  shipbuilding  acquisition  process 
and  tracked  until  desired  cost  targets  have  been  achieved.  In  addition,  iterative  CATV 
analyses  can  aid  Navy  efforts  to  properly  balance  cost  with  performance,  design,  and  system 
requirements  in  its  shipbuilding  programs.  Such  analyses  can  help  the  Navy  quantify  and 
evaluate  the  degree  to  which  its  planned  investments  will  result  in  increased  utility  to  the 
warfighter  and  subsequently  position  the  Navy  to  tailor  future  ship  capabilities  in  ways  that 
maximize  value. 


ARMY:  Future  Combat  Systems  fFCS') 

1.  Was  the  Army’s  decision  to  begin  System  Development  and  Demonstration  phase 
with  over  75  percent  of  FCS  critical  technology  elements  immature  consistent  with 
DOD  policy? 

The  decision  to  enter  the  System  Development  and  Demonstration  phase  with  75  percent  of 
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technologies  immature  does  not  satisfy  the  basic  tenets  of  DOD  policy  and  best  practices. 
DOD  policy  and  best  practices  are  very  clear  that  technologies  should  be  mature  before 
system  development  begins.  FCS,  in  particular,  is  worrisome  because  it  is  now  three  years 
past  the  beginning  of  development,  and  most  of  the  technologies  remain  immature  by  best 
practice  standards. 

2.  In  retrospect,  what  should  have  been  done  to  bring  the  program  in  line  with  DOD 
policy  and  best  practices? 

The  program  should  not  have  entered  System  Development  and  Demonstration  with 
undefined  requirements  and  immature  technologies.  Rather,  it  should  have  remained  in  the 
science  and  technology  community  until  the  enabling  technologies  were  mature,  or 
requirements  should  have  been  traded  down  to  match  the  available  technologies. 

3.  What  are  the  implications  for  other  acquisition  programs — will  they  adhere  to  DOD 
policies  for  awarding  System  Development  and  Demonstration  contracts  for  only 
those  weapons  whose  critical  technologies  have  achieved  desired  maturity  levels? 

Programs  like  FCS  underscore  the  challenge  in  getting  better  program  outcomes.  Despite 
being  out  of  alignment  with  the  knowledge-based  approach  outlined  in  DOD  policy  and 
advocated  by  best  practices,  the  program  was  approved  to  begin  the  System  Development 
and  Demonstration  phase  and  funded  at  high  levels.  The  example  of  success  then  becomes 
not  programs  that  follow  the  intent  of  the  policy,  but  those  that  do  not  and  still  win  approval 
and  funding. 

4.  What  can  be  done  to  mitigate  risks  in  FCS,  given  where  it  is  today? 

We  have  recommended  that  the  Secretary  of  Defense  limit  DOD’s  commitment  to  the  FCS  product 
development  phase  and  eventual  production  until  a  sound  business  case  that  is  consistent  with  DOD 
acquisition  policy  and  best  practices  can  be  clearly  demonstrated.  Further,  we  have  recommended 
a  go/no-go  decision  be  made  in  2008  after  the  preliminary  design  is  done.  This  would  in 
essence  be  the  time  that  the  program  is  ready  for  a  system  development  decision,  as 
technologies  should  have  reached  a  basic  level  of  maturity  and  system  level  requirements 
should  be  finalized  by  this  time.  In  addition,  the  Army’s  fiscal  outlook  is  extremely  challenging. 
Outside  of  the  Army,  competition  for  production  dollars  will  be  great  as  both  the  Air  Force 
and  Navy  have  significant  production  programs  planned  for  that  time.  Outside  DOD,  there 
will  be  an  increasing  squeeze  on  discretionary  funds  (weapons  are  discretionary)  from  other 
federal  sectors. 

5.  What  does  the  reliance  on  a  lead  systems  integrator  (LSI)  say  about  the  Army’s 
ability  to  manage  a  program  of  this  magnitude? 

The  Army’s  reliance  on  a  LSI  suggests  that  the  FCS  is  so  ambitious  in  scope,  complexity  and 
schedule,  that  it  exceeded  the  Army’s  own  ability  to  effectively  manage  such  a  complex 
program. 
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6.  What  risks  does  the  use  of  an  LSI  pose  for  protecting  the  interests  of  the 
government? 

The  use  of  an  LSI  can  pose  a  risk  with  respect  to  organizational  conflicts  of  interest.  For 
example,  an  LSI  might  favor  its  own  solutions  over  other  competing  solutions.  Further,  if  the 
LSI  stands  to  benefit  from  the  continuation  of  a  program  into  production,  it  has  a  financial 
stake  in  the  outcome  that  could  compromise  judgment.  Over  time,  the  government’s  ability  to 
oversee  and  act  independently  from  the  LSI  could  weaken  as  experienced  government 
managers  retire  and  dependence  on  the  LSI  increases. 


ARMY:  Warfighter  Information  Network-Tactical  f  WIN-T  1 

1.  Was  WIN-T  another  program  that  was  approved  with  unrealistic  expectations  for 
requirements,  technology,  cost,  and  schedule?  Did  the  program’s  original  strategy 
follow  DOD  policy  and  best  practices? 

At  the  time  DOD  approved  the  start  of  system  development  in  August  2003,  only  three  of 
WIN-T’s  12  critical  technologies  were  close  to  full  maturity.  Although  DOD  policy  calls  for 
technology  maturity  by  the  start  of  system  development,  DOD  did  not  require  that  the  other 
nine  critical  technologies  reach  a  level  close  to  full  maturity  until  the  start  of  low-rate 
production.1  The  technical  and  program  risks  of  moving  forward  with  immature  technologies 
are  exacerbated  by  the  significant  interrelationships  among  WIN-T’s  critical  technologies.  For 
example,  mobile  communications  technologies — a  critical  component  of  WIN-T’s  operational 
concept — rely  on  antennas  and  other  technologies  to  achieve  their  performance  objectives, 
and  a  lag  in  the  development  of  any  of  these  technologies  would  result  in  a  lag  in  the  overall 
development  of  mobile  communications. 

With  respect  to  requirements,  DOD  had  approved  an  operational  requirements  document  for 
WIN-T  shortly  before  the  start  of  system  development.  However,  WIN-T’s  interdependencies 
with  other  programs  further  heighten  the  technical  challenges  and  risks  associated  with  the 
program.  In  particular,  at  the  time  WIN-T  system  development  began,  the  Army’s  networking 
requirements  for  FCS — which  WIN-T  is  expected  to  heavily  support — were  not  well 
understood.  As  a  result,  the  networking  needs  of  FCS  and  the  capabilities  WIN-T  planned  to 
deliver  to  this  program  were  not  well  synchronized.  In  addition,  WIN-T’s  tightly  compressed 
schedule  assumed  nearly  flawless  execution  and  did  not  allow  sufficient  time  for  correcting 
problems  if  they  arose. 

2.  What  steps  did  the  Army  have  to  take  to  buy  JNN?  Why  was  this  approach 
necessary? 

Rather  than  undergoing  the  normal  acquisition  processes,  the  Army  resorted  to  extraordinary 
measures  outside  of  the  normal  requirements,  acquisition,  and  budgeting  processes — 
including  the  supplemental  budget  process — to  rapidly  acquire  and  field  the  Joint  Network 
Node-Network  (JNN-N).  Specifically,  instead  of  initiating  a  formal  acquisition  program,  the 


‘in  mid-2003,  the  Army  developed  risk  mitigation  plans  for  the  nine  immature  technologies,  but  a  program  review  sponsored  by 
the  Army  in  July  2004  concluded  that  these  plans  lacked  sufficient  detail. 
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Army  utilized  existing  contracts,  originally  established  for  modifying  legacy  networking 
systems,  to  procure  JNN-N  equipment  in  approximately  5  months.  For  example,  a  purchase 
agreement  from  the  WIN-T  program  was  used  to  procure  satellite  trailers;  a  General  Services 
Administration  contract  was  used  to  arrange  training;  and  existing  Army  contracts  were  used 
to  procure  JNN-N  in  seven  spirals,  with  each  spiral  equating  to  approximately  one  division’s 
worth  of  equipment.  Supplemental  funding  totaling  $763.6  million  for  fiscal  years  2004  and 
2005  was  used  to  purchase  the  first  seven  spirals  of  JNN-N.  Additional  supplemental  funding 
totaling  $818.7  million  for  fiscal  year  2006  is  being  used  to  acquire  JNN-N  lots  8  through  10. 2 

The  Army’s  approach  for  acquiring  JNN-N  was  intended  to  address  an  urgent  need  identified 
during  Operation  Iraqi  Freedom  and  Operation  Enduring  Freedom  to  provide  better 
communications  capabilities  between  soldiers  fighting  out  of  sight  of  one  another.  DOD’s 
processes  for  approving  formal  programs  are  extensive  and  have  not  been  conducive  to  rapid 
acquisition. 3  However,  these  processes  exist  to  ensure  that  programs  are  managed  and 
overseen,  properly  funded,  and  adequately  tested  prior  to  fielding.  The  Army’s  decision  to 
acquire  the  capability  through  an  approach  that  was  outside  these  processes  did  not  address 
the  legal  and  policy  requirements  for  managing  an  acquisition  program  of  this  magnitude.  In 
particular,  JNN-N  procurement  did  not  complete  initial  operational  testing  before  going 
beyond  low-rate  production,  as  required  by  law  and  DOD  policies. 

The  acquisition  of  JNN-N  through  extraordinary  measures  was  made  necessary,  in  part,  by 
DOD’s  decision  to  acquire  a  revolutionary  WIN-T  capability.  When  the  Army  opted  to  pursue 
large  technology  advances  in  networking  capabilities  to  support  the  future  force  through 
WIN-T,  rather  than  pursuing  a  more  incremental  approach,  it  accepted  a  gap  in  providing 
tactical  networking  capabilities  to  the  warfighter.  Moreover,  the  Army’s  optimistic 
acquisition  approach  for  WIN-T  created  the  impression  that  this  capability  gap  was  far 
smaller  than  it  really  was.  As  the  Army  pursued  an  advanced  WIN-T  capability,  its  existing 
communications  equipment  became  increasingly  obsolete  and  the  need  for  near-term 
improvements  became  more  urgent  with  the  onset  of  the  war  in  Iraq.  If  the  Army  had 
followed  DOD’s  acquisition  policy  preferences,  which  emphasize  achieving  capabilities  in 
increments  based  on  mature  technologies  to  get  capability  into  the  hands  of  the  user  more 
quickly,  it  might  have  been  able  to  get  needed  communications  capabilities  to  the  warfighter 
sooner. 


“In  keeping  with  its  transition  from  a  division-based  to  a  modular  brigade-based  force,  the  Army  is  now  procuring  JNN-N  in 
batches  of  brigade  sets.  It  tenns  these  batches  lots,  rather  than  spirals. 

3The  Army  explored  using  special  rapid  acquisition  authority  to  respond  to  combat  emergencies  to  formally  waive  acquisition 
laws  and  policies  for  JNN-N,  but  the  dollar  limit  for  use  of  this  authority — $100  million  per  year  for  the  entire  Department  of 
Defense — was  far  too  low  to  accommodate  the  Army’s  need  for  JNN-N.  See  section  806  of  the  Bob  Stump  National  Defense 
Authorization  Act  for  Fiscal  Year  2003  (P.L.  107-314),  as  amended  by  section  811  of  the  Ronald  W.  Reagan  National  Defense 
Authorization  Act  for  Fiscal  Year  2005  (P.L.  108-375). 
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3.  What  lessons  do  the  WIN-T  and  JNN  acquisitions  have  for  DOD’s  requirements, 
budgeting,  and  acquisition  processes? 

While  JNN-N’s  acquisition  was  a  pragmatic  response  to  an  urgent  need,  the  urgency  was 
brought  about,  in  part,  by  the  Army’s  decision  to  use  a  revolutionary,  rather  than 
evolutionary,  approach  to  acquiring  WIN-T.  This  approach  appeared  less  risky  at  the  time 
because  optimistic  projections  created  the  impression  that  the  capability  gap  was  narrower 
than  it  really  was.  In  hindsight,  focusing  on  meeting  the  needs  of  the  future  force  at  the 
expense  of  providing  capabilities  to  the  current  force  proved  detrimental  once  operations 
commenced  in  Iraq.  To  field  capabilities  quickly,  the  Army  did  not  use  DOD’s  formal 
requirements  setting,  budgeting,  and  acquisition  processes,  and  over  a  billion  dollars  was 
spent  on  JNN-N  without  it  being  established  as  a  formal  program.  This  is  not  a  good  model 
for  providing  needed  capabilities.  The  WIN-T/JNN-N  experience  has  implications  beyond 
these  two  communications  systems.  The  Army’s  experience  thus  far  creates  opportunities  to 
capture  lessons  learned  and  apply  them  not  only  to  the  Army’s  two  programs  but  also  to 
DOD’s  larger  ongoing  efforts  to  reform  its  acquisition  processes.  We  recently  recommended 
that  the  Secretary  of  Defense  examine  ways  to  apply  the  lessons  learned  from  the  Army’s 
experience  with  the  acquisition  of  WIN-T  and  JNN-N  to  ongoing  DOD-wide  efforts  to  reform 
its  key  decision  support  processes  for  setting  requirements,  allocating  resources,  and 
acquiring  weapon  systems.  DOD  concurred  with  this  recommendation. 

4.  What  challenges  are  presented  by  the  merging  of  aspects  of  the  WIN-T  and  JNN 
programs? 

The  Army  ultimately  intends  to  create  a  tactical  communications  network  that  functions  as  a 
single  tactical  backbone.  To  achieve  this,  the  Army  must  now  reconcile  two  separate 
acquisition  efforts  intended  to  meet  one  overarching  need.  Without  a  sound  strategy  for 
meeting  near-  and  long-term  needs  for  communications  and  networking  capabilities, 
significant  risks  remain.  Affordability  continues  to  be  an  overriding  concern,  particularly  as 
the  Army  and  DOD  as  a  whole  struggle  to  meet  the  needs  of  current  military  operations  while 
making  prudent  investments  in  the  future  in  a  difficult  fiscal  environment.  We  recently 
recommended  that  the  Secretary  of  Defense  direct  the  Secretary  of  the  Army  to  develop  a 
sound  strategy  for  transitioning  the  Army’s  tactical  communications  from  JNN-N  to  WIN-T. 
Such  a  strategy  needs  to  ensure  that  requirements  for  both  of  these  efforts  are  clearly  defined 
and  well  integrated,  that  the  fielding  strategy  is  executable,  and  that  sufficient  funding  has 
been  committed.  DOD  concurred  with  this  recommendation  and  noted  that  the  Office  of  the 
Secretary  of  Defense  has  directed  that  an  approved  JNN-N  to  WIN-T  transition  strategy  be  in 
place  before  acquiring  additional  JNN-N  systems. 

Air  Force/NAVY/Marine  Corps:  Multi-Service  F-35  Joint  Strike  Fighter  ( JSFj 

1.  Were  JSF  technologies  mature  at  the  time  of  the  decision  to  enter  System 
Development  and  Demonstration? 

JSF  entered  development  without  its  eight  critical  technologies  being  mature.  Information 
from  the  program  office  indicates  that  maturity  has  progressed.  However,  seven 
technologies  were  not  mature  at  the  recent  design  review  and  two  of  the  technologies 
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needed  for  mission  effectiveness  are  not  expected  to  be  fully  mature  until  3  years  after 
production  begins.  We  reported  in  October  2001  that  failure  to  mature  these  critical 
technologies  could  result  in  increases  in  the  production  and  long-term  ownership  costs, 
schedule  delays,  and  compromised  performance  as  problems  arise  in  product  development. 

2.  Is  the  program  ready  for  a  low-rate  initial  procurement  decision  in  FY  2007?  What 
risks  does  the  program  face  at  the  time  of  this  decision? 

The  JSF  program  is  not  well  positioned  to  enter  production  at  this  time.  At  the  time  of  the 
low-rate  production  decision,  only  one  early  prototype  will  have  flown  and  less  than  one 
percent  of  the  flight  testing  will  be  complete.  In  addition,  none  of  the  three  variants  will  have 
a  production-representative  aircraft  built  and  in  flight  testing.  The  program  intends  to  invest 
$26  billion  in  initial  production  before  a  fully  integrated,  production-representative  aircraft 
will  fly  in  2011 — 4  years  after  the  production  decision.  The  cost  of  discovering  design 
problems  during  production  could  be  significant  if  testing  shows  that  large,  structural 
components  of  the  aircraft  require  modifications.  Furthermore,  the  length  and  scope  of  the 
remaining  effort  make  it  difficult  to  accurately  estimate  cost  and  completion  schedules.  The 
program  has  about  7  years  to  complete  the  remaining  activities  of  the  System  Development 
and  Demonstration  phase,  such  as  developing  22  million  lines  of  software  code  and 
completing  a  7-year,  12,000-hour  flight  test  program.  By  investing  heavily  in  procuring  JSF 
aircraft  before  flight  testing  proves  it  will  perform  as  expected,  DOD  has  significantly 
increased  its  risk  of  adding  more  cost  and  delaying  the  delivery  of  critical  capabilities  to  the 
warfighter. 

3.  What  criteria  must  the  program  meet  to  demonstrate  readiness  for  low-rate 
production? 

Key  indicators  of  a  program’s  readiness  for  entering  production  and  making  significant 
investments  in  tooling,  facilities,  and  materials  include  demonstrating  that  (1)  the  aircraft’s 
flying  qualities  function  within  the  parameters  of  the  flight  envelope,  (2)  the  aircraft  design  is 
reliable,  or  (3)  a  fully  integrated  and  capable  aircraft  system  can  perform  as  intended.  In 
addition  to  a  stable  design,  production  processes  must  be  mature.  This  point  is  achieved 
when  the  program  can  manufacture  the  aircraft  within  cost,  schedule,  and  quality  targets. 
Because  the  program  will  lack  key  design  and  testing  knowledge,  DOD  plans  to  use  cost 
reimbursement  contracts  for  early  production  aircraft.  This  type  of  contract  places 
significantly  greater  cost  risk  on  DOD  and  the  taxpayer.  Confidence  that  investment 
decisions  will  deliver  expected  capability  within  the  cost  and  schedule  goals  is  likely  to 
increase  when  flight  testing  proves  the  JSF  will  work  as  expected. 

4.  When  will  the  first  production-representative,  fully  integrated  prototype  aircraft  be 
demonstrated?  How  many  production  aircraft  will  be  on  contract  by  that  time? 

Under  the  current  schedule,  the  first  production  representative,  fully  integrated  aircraft  will 
not  begin  flight  testing  until  the  2011  timeframe.  This  occurs  4  years  after  production  has 
begun.  At  that  time,  DOD  will  have  committed  to  buy  about  190  aircraft  costing  $26  billion. 

By  the  time  testing  is  planned  to  be  completed,  in  2013,  DOD  will  have  procured  more  than 
double  that  amount — 424  aircraft — at  an  estimated  cost  of  about  $49  billion. 
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5.  What  are  the  cost  implications  if  that  prototype  indicates  the  need  for  design 
changes  at  that  stage  of  the  program? 

The  costs  of  discovering  design  changes  could  be  significant  if  testing  shows  that  large, 
structural  components  of  the  aircraft  require  modifications  or  mission  systems  require 
additional  development.  For  example,  problems  discovered  late  in  the  development  of  the 
F-22A  increased  a  4-year  flight  test  program  to  about  8  years  and  affected  the  program’s 
ability  to  conduct  operational  testing  and  move  into  production.  In  the  JSF  program,  design 
changes  needed  in  one  variant  could  also  ripple  through  the  other  two  variants,  reducing 
efficiencies  necessary  to  lower  production  costs  with  common  parts  and  manufacturing 
processes  for  the  three  variants.  Some  industry  officials  have  indicated  that  the  cost  of  design 
changes  such  as  these  could  be  10  to  1,000  times  greater,  depending  on  how  far  the  product 
has  progressed  into  production. 

6.  Under  a  cost-reimbursable  production  contract,  will  the  government  be  responsible 
for  assuming  the  risk  and  cost  of  any  needed  design  changes? 

Cost  reimbursement  contracts  provide  for  payment  of  allowable  incurred  costs,  to  the  extent 
prescribed  by  the  contract.  These  contracts  are  generally  used  when  uncertainties  involved  in 
contract  performance  do  not  permit  costs  to  be  estimated  with  sufficient  accuracy  to  use  any 
type  of  fixed  price  contract.  Under  a  cost  reimbursement  contract,  the  government  assumes 
most  of  the  risks  of  performance — because  it  agrees  to  pay  the  contractor  its  allowable 
incurred  costs  to  the  extent  prescribed  in  the  contract. 


Air  Force:  F-22  Aircraft 


1.  What  fundamental  causes  led  to  the  F-22’s  cost,  schedule,  and  performance 
problems? 

We  have  reported  in  the  past  on  the  inefficiencies  of  the  F-22  acquisition  program  that  led  to 
numerous  schedule  delays  and  cost  increases.  This  program  provides  an  excellent  example 
of  what  can  happen  when  a  major  acquisition  program  is  not  guided  by  the  principles  of 
evolutionary,  knowledge-based  acquisition.  The  F-22  program  failed  to  match  requirements 
with  resources  and  make  early  trade-offs  and  took  on  a  number  of  new  and  unproven 
technologies.  As  a  result,  the  program  failed  to  capture  the  appropriate  technology,  design, 
and  manufacturing  knowledge  of  the  F-22  aircraft  at  the  right  times  during  the  development 
process.  This  created  a  challenging  and  risky  acquisition  environment  that  ultimately  delayed 
by  about  10  years  to  the  warfighter  the  capabilities  expected  from  this  new  aircraft.  It 
additionally  led  to  rising  development  costs,  affordability  issues,  and  changing  requirements 
over  the  19  years  in  development. 
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2.  Has  the  F-22  followed  DOD  policy  and  best  practices? 


No,  the  F-22  did  not  follow  a  best  practices  approach  to  its  development  and  procurement. 

As  to  whether  the  F-22  followed  DOD  policy,  we  have  not  specifically  assessed  the  F-22 
program’s  compliance  with  DOD  policy.  However,  we  reported  in  2003  t GAQ-04-53)  and  2006 
t GAQ-06-368)  that  DOD  policy  includes  a  good  framework  for  weapons  system  development 
and  acquisition,  but  it  does  not  include  sufficient  controls  to  ensure  key  elements  of  the 
policy  are  implemented  in  each  phase  of  a  weapons  system  program.  As  a  result,  our  2006 
report  found  that  weapons  system  development  programs  continue  to  experience  cost  and 
schedule  growth. 

3.  What  concerns  do  you  have  over  the  multiyear  procurement  proposal  for  buying 
more  F-22s? 

In  testimony  earlier  this  year  before  the  Airland  Subcommittee  of  the  Senate  Armed  Services 
Committee,  we  expressed  a  number  of  concerns  about  an  Air  Force  proposal  for  multiyear 
procurement  of  the  F-22A  ( TACTICAL  AIRCRAFT:  Questions  Concerning  the  F-22’s 
Business  Case,  GAQ-06-99 IT.  July  25,  2006).  These  concerns  related  to  the  savings  expected 
to  be  realized,  the  use  of  incremental  funding,  and  the  cost  risk  that  could  arise  if  quantities 
to  be  ordered  continued  to  decline..  Although  Congress  subsequently  authorized  multiyear 
procurement  of  the  F-22A,  it  listed  a  number  of  requirements  the  Air  Force  would  have  to 
address  related  to  the  concerns  we  had  expressed.  In  addition,  the  Congress  expressly 
prohibited  the  use  of  incremental  funding. 


Questions  from  Representative  Kaptur: 

In  addition  to  the  questions  listed  above,  the  Subcommittee  also  submitted  to  GAO  and  DOD  a 
list  of  eleven  supplemental  questions  for  the  record  from  Representative  Marcy  Kaptur,  some  of 
which  deal  with  the  use  of  contractors  in  Iraq.  These  questions  and  others  touch  on  an  emerging 
issue  of  critical  importance,  that  is,  the  increasing  role  that  contractors  play  in  conducting  the 
business  of  government.  We  appreciate  and  share  the  concern  underlying  the  questions. 
Although  DOD  is  in  a  better  position  than  GAO  to  provide  the  information  that  would  be  needed 
to  respond  to  each  of  the  eleven  questions,  we  have  issued  reports  regarding  the  use  of  private 
security  contractors  in  Iraq  and  DOD’s  use  of  logistic  support  contractors,  which  we  believe  are 
relevant  to  the  issues  raised  by  Representative  Kaptur.  The  reports  are: 

REBUILDING  IRAQ:  Actions  Needed  to  Improve  Use  of  Private  Security  Providers, 
GAQ-05-737.  July  2005. 

MILITARY  OPERATIONS:  DOD’s  Extensive  Use  of  Logistics  Support  Contracts  Requires 
Strengthened  Oversight,  GAQ-04-854.  July  2004. 

We  would  be  pleased  to  provide  copies  of  these  reports  upon  request.  They  are  also  available  on 
our  Web  site,  http://www.  gao. gov. 
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